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ABSTRACT , - \ 

• /» 

This paper .describes the, development and -first administrations of a survey ^ 

A 

of attitudes toward a large number of policies and procedures in the evaluation o'i 
* classroom teaching. The instrument; developed cooperatively by administrators, 
^teachers, and consultants, asked how Itmportant ^ach policy or procedure was iijv an 
ideal evaluation system and then how well it was being handled locally • Two Milwaukee 
CQ area school districts were surveyed. Results showed some discrepancies between, real 
'and ideal systems and some group differences. Teachers definitely wanted to be 
involved at all stages. All stages jii/^re^seen as being important. Methodological ' 
CX< problems are discussed and some revisions suggested. 



The evaluation of instruction has been a thorny issue since ancient times. 

Even Aristotle, in about 347 B.C., at Plato's Academy was passed over when Plato 

died and the head positi6n went to someone else. 'Aristotl^, while departing for 

greener -pas tures , must haye had some serious doubts about the fairness of the 

evaluation job doire on himi ^ ^ , , 

■ Of course we all re-conginze that there are still many problems in the 

evaluation of^teach'ing today. This is not only b.ecause the evaluator's task is 

. * # 

a complicated and delicate one, but also because there are so many different 



CO 



* Barry Fagin passed away on July 5, 19/^-. As with many other projects, he 
got this one moving and then handed it over to some interested friendg to 
finish while he turned his attention to new problems.' 



t 



- ''<• 

opiniWiS about wriat "guod ^tedchmg" really «is^ . . ^' 

• " ■ ' - • - / 

. Atiothe^" problem i.l^ac ovalaH>^>rts or ^teacMng nay 'soon be ^^equired to 

deal with is new state legis,lation on the* evaluation of educators. California's 

Stull Act is the''fir^D,guch legislation we have heard of/ It requires ,each scfiool • 

district ifi California to ^adopt or^ develop its-^own sys-tem of evaluating certificated 

^ \ - ' • ^ ' " 

pei;sonnel. It ds presented along with eigUt inajor presentations plus some thoughtful' 

* * / ' 

discussion? in-a conference report edited.by N. L. Gage' (1973). 

There have been' developed some workshop materials on the evaluation of teach- 
ing during the ikst three years. One of these, also from California (no coincidence), 
was developed at the Center for the Study of Evaluation at U.C.L.A* (Fink, e_t, al> , 
1973). It has recently been adopted fo*r use in national workshops by the National 
Symposium^ for Professors of Educational Research. It covers': Teacher observations 
and ratings, Teacher skill testing, and Measurement of student growth ^as three^general 
options in the evaluation or appraisal of teachers. . -a 

But this kind of workshop was probably forced by the Stull Act. Would it not 
be more advisable to find out what!' educators t^in'k is valuable before deciding upon 
one or more of the three options described above? ^ ,^ 

In- the present X^^Y we^tried to get a better idea of how school people per- 
ceive the '^e valuation process. We wanted to present a large number of specific poli- 
gies and procedures and get judgements about^how important or valuable they yould bfe 
an ideal system of evaluation. Then if the important policies were not being 
.practiced in a dist^rict or building, the people who were responsbile could^ initiate 
those policies. Likewise, they could discontinue any policies or procedures seen as 
unimportant or trivial. ^ - . 

♦ 

We were also interested in any policies or procedurt-cj for which there were 

* d^isagreements between groups as to their importance. It seemed likely th^t admin7 

. ' istrators and teachers would see some items dif f aren-^tl'y . Tpe ,s^me, for elementary 

* ^ f • 

versus secondary educators since Lheit teaching tasks are raLher different. Finally^ 



Q teachers who had tenure might nave - dif fcereqt needs and ya.lue.b from those of non-tenured 

ERIC . ,o 



teachers. - * 



Peth^ps Wfc luukcid v^iLh the must ho^pe tor those poll ciesl and prqcedures which.& 
were jmlged to te impuitdnt dcrosb all gruupo of school p^ple| These, we thought, 



mi^ht be th\ elements of a newer and better system- of evali^atioli. 

Procedures ty^' ' ^ . . . \ ^ K 

A surVe^ ins,trunienc was developed in a cooperative pro^j^'ctlin which twelve 
public school districts in Milwaukee and Ozaukee caunties particp^ted. This 'pro- 

■ ■ ■ ■ " A ' ■ • 

ject was initiated by the late^JBarfy Fagin^ * He and four /school teachers met monthly 
during the 19 72-73 schpdl year and the otfier co-authors joined the project at ?jario.us 
times after that. ' \ ^ ' , . 

During, the Fall and Winter of 1973-74 the project was formalized -^and its 
efforts were focused upon constructing and ^piloting the survey. The survey was 
developed in a coope'Tative and ir\teractive 'fashion. There^ were five* preliminai 
forms of the ^survey that were developed, printed, studied, critiqued, defended,| 
and revised by the participants before 'the pjresent version was printe4..^ Tl^ entire 
survey 'insttument is '^ttach^d. \ • ■' r" '-' J-' V' 

*This survey is unique ^n three i;eap^ects. First, it ia' tjijii-only survey we 

^' ' ^ r'^^ ' ■ M#t'>' ' A 

know of that assesses^ attitudes toward specific evaluation of ^ceachi^ policies^ 
Second, it is veiry comprehensive' in scope; we asked^ the respxjhde^t to react 1^ 

all the detailed .issues that/seemed to' have spme^,men^t'^ ThiM,; it allows for ^'t 

^ ' ' < ' / ' / ' . " 

comparisons between how Important a policy or procedure is and how well it is 

, * • \ ' ' ' - 

' . ' ' " ' /• '^f 

being handled locally. , ' . * ^ ' * .V , 

Its. great comprehensiveness was seen as a necessary inconvetiience , at J^^st 

-on the intial administration^,. Unimportant items could be founci empirically ^J^p^ 

then be removed, for subsequent revisions.' Seven' operational phases, or steps 

the evaluation .of .classroom teaching were designated' in the survey, . They weW J:' . 

arranged in a .tempor'aCl , order With three ^planning step«, three action steps, 

- ' " ' \- ' - A ' ■ - ' - , ^. ' ' 

one feedback, step. \- These, seven, are 'listed btlow With an eicplan.iition for. each' one; 



Somewhat ^briettti -expioaaLiou£> Veie givtn to th^ res^pondentb in ,the seven corresponding 



sections of the .bUtvey icseit^ Tht auiubtir ui i>pecifiN: itc^ms for each of the-seven 

/ • ' * - 

sections is indicated in parentheses. 

-A* Selecting' and Defining -Educational ObjectiveS4 > This section dealt 

with issues such as who should be responsible for specifying objectives or goals 
* • ^^-s 

toward which teachers Should aim and how should those objectives be* determined? 

Fo^r examjple, one objective might be, *'Each teacher will provide for .Individual 

differences/* (14 items) ^ , ' ° 

Determining Variables to be Employed in Evaluation ^ While Section A 

dealt with broad objectives or goals, Sect^ion dealt with the manifestations .or 

measurable overt behaviors which are associated with tho^e goals • An example 

' ' , ' \ ^ 

under the broad objective about providing for individual differences ,in A might 
be, "Bach teacher should make available instructional materlaj.s ,which provide 
alternative reading difficulty levels. "T Both process* and product evaluation were 
considered. (13 items) ' ^ . ' • 

C. Determining the Technique to be used. in Gathering Evaluative ^Information . 
The main issue here was whether the evaluation' of teaching should be based upon 
observational techniques or upon more objective techniques where some kind of survey 
or te^t instrument would be used. (-13 items) 

~~ D. Guidelines for Observational Assessment . Assuming that ^somfe kir^ <j(f 
observational procedures would be used, who should determine the procedures. 



ho 



would 'do the observations, and what form should the observations take? (>25" items) 

\e. Guidelines for Instrumental Assessment s Assuming that some kind of 
instrunents (achievement .teste,, attitude surveys, etc.)^ were** to be used, who should 



ponsible to select or develop those instruments and what form -should they 



1- 
,e? (12 items) 



^ Deciding about Standards of Excellence^ in Teaching' ; Who should be 

responsible fot setting up standards .of excellence and how would* these standard^ 
be applied^^'Un' the whole evaluation system* (ll'Jtems) 



• G,^ How Should Evaluation Infonnatictn be Shared ? This section dea}.t with 
* • # 

feedback sessions. It was assumed that the flow o-f information ^might go both 
wdys between teacher and evaluator and not just the usual one-way flow from the 
evaluator to the teacheri (8 items) ^ . ' 

^The respondents were aa^ed first to rate^.each item on the basis of its 

Importance on a 5-point, scaleSfrom 1 ("Least Import^i") to 5 ("Most Important")**^ 

: ' - , ' . ' • « ' 

If they thought Chat the i.tem "Should not eveji be considered," they were asked to 

A 

signal this by marking an X instead of a^^number. Secend, for %he same item, , they 

were aSked to indicate how Effectively the policy or procedure had been handled ♦ 
J ' ' ^ ^ ' ' * ' ' 

•^in theit own school's ejValuat^on system. Again a^5-point scaj.e waS used from 1^ 
* ° ' ' 

("Ineffective") to 5^ (^ff ective")^ If the policy or procedure was not, being 

practiced^ In "^their system, they were* asked to signal this fact by^marking an X 

agadn instead of a number, t . . 

A pi:oblem was anticipated by some of our participants*. It seemed very • 

\ <t , ' ' ^ , / ' • 

likely that teachers would give high Importapce ratings" to some ^valuation . ^ 

.1 ^» . ^ * - * ; , ' ' / . 

"practices if they were to be used purely for the improvement of instruction but 

definitely *not' if those practices might subsequently be |Used against the teachers. 

Mpreover, it would not be fair to advance an evaluation system for administrative 

decisions like promotion - non prombti^jii retenti*on or .dismissal, merit pay, work 

assignments etc. when the evaluation system had been planned by teachers who 

■' ■ >^ - • ^ ■ • • ' 

thought it was going to' be used for improving instruction. In .order to overcome 
this problem, we had ^each respondent take |:he entire .seven sections (96 items) two 
times. The first timej^hey were to assume that their .ratings would be used to 
*^hape an evaluation system purely for the improvement of ijas^truction. The second 
_jtime ,they.„were,^.to assume that the eventual _system would be used fq^ t^e Winds of 



, administrative personnel de^cisiens mentioned above. 



final section- of the survey, labeleW "Par-t.II", presented '.eight ways in 
which evaluations, of teaching could be used, for exampie; to improve instruction 
or to reward superior teaching performances financially. • ' - - 



6 - , 



^ " . There, was <i t*'/:.^^ of 200 it-^!<ib.. 0^ rlu^^V. 19||pir^d lu do wi'th evaluation 

policies and prnt:t-<^'ire- i ' < ^.^^u* I t puip^sc^s ot evaluation of - Jieachinge 

J ' \ ^ * ' * < ' * '''' 

The 192 figure represents 96 ittino taken under i.'o asbumptions . Since each item was 

rated* with reference to both Impoi taRc'e ^id Et tt ctiveness , each respondent made »400 

• • separate ratings ♦ / / 

The respondents were, f com two suburban Milwaukee 'school, 'districts. In pne 

, % f.Ji^ district, to, which We will_ givef -the pseudonym, IJixchwood, all teachers, adminis- 
S * ' ■ 

trators, 'Supervisors, and specialises responded. In the second district, to which 

. ' ^ ' . , ' • . • ' .^"^ , y \ , ^ 

we will give the pseudonym, Pine Mountain, only the teachers *^nd .administrators d'f-' 

/ ' : - ^ ' - ^ . , 

the elementary scKools responded The^t;o}tal N*s were 134 and 74 respectively. ' x\ 
' ' : / * ^ . , 

/ ' • ' ' 

Since the response sheets were not optically ^Jcanable, we had all demographic 



and^item response data keypunched. A computet program called CObfREQ (part of STAT- 

M ^ ■ • ■ ' . • ■ 

JOB) was used whi^^h reported the percentages of the ^roup of respondents who g^ve 
each item ratings of>}., 2, 3, 4; 5^ and X, (keypunched 'as/- a blank). It alsp gave- the 



means for ea<ih *item. 



, / \t ^ij^uld be rememtif^red ph^t on any item some respondents marked an.OC for 

either Importance oi^^^f f ecti^^ness or both- Accordingly, the mean ratings ^are based 

* ■ * . -I 

^nly on Che responses of those people who iriade^hufflericai responses, ^^ince X's we^re 

recorded as^a categprical-ly different response. As \a result, there is no stable 

group of people wh0se responses are &eflec;ted. in the item mean scopes and it is not' . 

■ ' ;■ ' . ° - /' ■ ■■ " ' • 

proper to use these .means in any stajiisti'cal test of the differences between items ^ 
'* ' » / * • • . • , ' 

{ • . ' . •* ' . ' . 

or of**t^he differTences between i;esDpndent groups- Nevertheless, mean ratin'gs were 

calculated as a.;'slngle, crude, index at Importance or ot Ef tectivenessV." They are 

presented in categorical form with* rather llarge class inter vals. Percentage data 

^were provided 'to the two ^districts but requite'd'*too, much ♦space", for this report. 

Results; : ' . ' 



A • . Differences between rat t n^^s o f 



. ' -While we hgd expected rar.her rrejcj^uent vi^lf 

^ assumprions *about how the e-VLntual system 

ERIC : .\r I . ' 7.. 



arnporranre uad^.Ji: the^'two' assumptions. 



erhT5\,e':» beti^/eerf^ ratiags- und^r the two 

u; uAaiuatiui* vould bu^- Uj^.ed ,' there w^re - 



almost no, differences observed. Out of 96 items, only* two items, and B/ ,'sh'bx>^ 
a noticable mean difference. Both of these were' given higher Importartce ratings 
under assumption //I than assumption ?/2. Both items proposed including* Parents in ^ 
the process of evaluation* There is some sense Jto these results, but, the mean dif- . 
ferences were not very large and were -were a large number of expecfed (differences 
that did not, "Show up. We concluded that any differences due to assumption //I 
versus #2 were few enough an'd small enough to ignore and- the results described 
below are based on responses under assumption //I only (for improvement^ of in-' ^ 
structipn) . ^ ' ' ' . 

" ' 2. Isolation of crucial^items and of rejected items • The item mean, ratings 

ft ' . ' K 

for the Pine Mountain re^ondents ire given in Table 1. The numbers in the cells 
refer, to specific items*, within the sev^n sections j A - G, and Part II, .which dealt 
with purposes of evaluating -teaching/ It is rather easy to spot itims which were ' W 
judged to be important or Unimportant in each Section, Note that section G really 
had no items that were Unimportant or even Moderate Importance, Note also that 
every sectioi>. except Part II had at least four items with mean ratings of High 
Importance, (ie. over 4.0). Since items with a small numbei: of respondents produced 
meaijis that ar^ somewhat suspect, those items are indicated with one or two asterisks. 
All But one of ^th^e (//D-|^y) had meai values below 3,0 vhich suggests that the mean * 

~ , ' ' ' ^" ^ ■ ' * . ^ ^ - 

values are not too misleading. 

^ 

(INSERT TABU 1 mOUJ HERE) • ^ 



3. Iso lation of items that are bejt^rg^^ell handled versus pocxrly handled 



Iftcally . The item mean ratings on Effectiveness^ are given for the Pine Mountain ^ 

district in Table 2.. At/first glance one is struck by the fAct that no „^em ^r^^ 

f . - ■<■ 

;^n Ef^ctiveness rating of 4.01 or higher. Eyen with a progressive' district there is 
room for .Improvement. 'An examidation of the three Pine Mountain schools ' '"individual ^ 
means showed a sprinkling o'f values abov^ the 4.01 level foi: one s^chool or another, 
however, SO the higher ratings'are within reach. « 
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* an'd •* vLftdicate iteias with 33% or aora a}id 67% or more ratings of •'X" , respectively. 



^ ' The second N/ery noticable £acc is chat there are a've^y large number of 'items 
that are reported as not .being used (hoc6 the one or two asterisks). We also can » 
see thajt there are some items that had medd Effectiveness^ ratings of 3.01 or hi-gher 

,but were also marked with an* X by many respondents (B3, C-^^ En^^^^r ^10^* 

» * *. 

Again, an inspection of the responses of the three .indivj-dual schaols accounts for 
tfie inconsistency. Usually two of the schools don't use the policy and the other 
onfe uses'^it .rather Effectively; thus, we have many X'9 and a ^mean of over 3.01* • 



IS 

-VN . 

' ^ ' ^(JNSERT Table' 2 abou^ here:) 



4/ Isolation of items- that ,are not being practiced locally^ T^ble 3 presents 
these data for the. survey, one section at a time, including- the verxf^^l^ci^^l^t Part II 
items\ If the item is^Qli the ri^ht most people reported that it was NOT b^ing practiced 
It is not too surprising to see so many items being frequently reported as not being 
practiced since we tried to J.nclude innovative policies andxprocedures -in the survey - 
especially in Sfectioft D, on the details of observational assessment techniques. 

It is surprlsir^g and ,a bit disconcerting to see so^many items in th^ middle 

ranges, from^il% to §Cf^ X's. At first glance it wqurld appear that the staff •'rfeally 

- - . , , - ^ . 

doesn't know what poli^cies are in effect and what' ones are not ' in 'affect . However^ 

there are several alternative explanations,. Perhaps the ite^i is being^^raqticed in 

one .building but not in others. Perhaps the item is being practiced on some res^pondents 
- ' - f . ' ' ' ' • 

(e.g. recently hired teachers) but not on others.^ ' Finally, it is possible that the 

item was poorly written and was easily misunderstood by the respondents. Thi§ was a 

particularly fruitful part of the analysis for the administrators sinc^ it suggested 

that they might need to make it. more.cliat to their teachers exactly^ y?hat' policies' 

• ' ' * i . /t . ' ' ^ * 

and practices. are in for|e. ^- \ . " f 



• (INSERT TABLE 3 ABOUT HERE) 
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The * and ** ma^ks indiWte items wiur 33% and 67% oi more ratings of "X". 
respectively. \ ' . 
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5 .*• Isolation br . discr''^pancl,fei5 -between gatW^I^ >b| 'Importahfie and ratings of . 
Effectiveress . -Only , the first sedtlon^ /^^eic^ccins *^ttd ^fejfii Educational Objec- 
* J tives,'o'is presentejd'herfe while all eighiTae^l^^enF^W^:^^ fashion , 

r!;0-: ti the'clistifict^. . J^i&ure^lrsfrows jthe _1A items of Section A witli. Iregard^o both-' 

, ,,.dieir m^an..Vaiuls /of. .judged Itnportance,Xl)9Jizontal; ax^s) and Ef fee£iveness (vertical 
' axi^y .; , One . dkshe4^./iin^^^]^r|2i^to*^ tfie inedian^^v^lue on one axis and the other dashed 
.'^^ . ..I'in^ repr(35erfti^;'th^^^ the otfteT^I ^ • /V ./J .f. A i '"'i ^"'^'*:'1..*V^'.'; 

y * -. Iif tife, upper left q^iadrant thus f orme'd?^;thefe^^t;^^^^ wfii'ch 
^ we" have' *somewliat_^rbiti?ari b^ing well him'dled'lo<?iIiy,'^'6,ut;^^^^ -..^ \ _ 

very" Important. -In the lower right 'quadrant thus formed, "there are.. two other items', ^ . 



13 and 14, whi<:h* we have Identified ^s heing. quite Important but not' verV well 
handled locally. ^ . ; ' * .... . . \ 

There are other way% of pointing^ut^uch, dilscrepancies biit.they ^re mgre 



complicated. For e^fample , ^finding 



fi^;^ V squar,es regression line is one good 
: > J. ^ . (rNSERT ?l 



he itemsK-^at d,epart the most from a least 
alternative, " v . - 



/ 



. (rNSERT FIGURE- 1 ABOUT HERE) . - ^ ^ 

V . * ' 6 •^^ .Recommenda^tions of whp shouid»b0 involved in the' seven phases of - ^ * 
' ^ evaluating instruction . Teachers and Administrators were given high endorsements 
in sj^ven phases. On the other, hand. Students, Community members, Parents, and 
Scnpoi* Board members were given .^uch lo\^ rating*s we oould conclude that they were 
rejectecf as jp'otenfiial partlcipairts in. all phases'.' , The teachers 'seeme a to be more 

^1*' interested in self *evaluati^n than having thfe:^r' colleagues observe an J evaluate 

• " > ^ • ^ f • /' 

them. Finally, Educational Ex'p^jcs were consistently given loedium r* ,ngs-'as .po,^ 

teiitial participants in itHe 'evaliiation brocesfs. ^ , . ^ ' /A ' 

- 7.. Ratings 6£--purposes d.f evaluatioh t The most Important purpose of . . ^ 

^C'*-. evaluating teaching was to Jielp ^teacher^ improve ^their teaching ef fectiven^s. 

This can be seen in Table 1. HdweVer, ip is also apparent that the other us*e$ , 

gj^Q of ''evaluating teachlri^ *are nrot :.|^;t«r4yialr*^ as^vve- had ^ticipated. 
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FIGURE 1. i5eM mean RATINGS. OF IMPORTANCE AN1> i;t*FECTlVENE 
FjOR THE - SECTION ON- SELECl'ING , AND DEFINING ED(j6ATlgN7VL 
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" * • 8.' Dtfrerences/Detweeen e-rbuns^ Al*l of the follbt^jitiff 't'esulC's Were observed*. 



8.' D^f ^ ranees /^S^we^n' gxbups^ >. of the follwilig*|esul 

h the;;&ii^lj^.^ts>^:;Bi^"a^ 4at:>a.' :-^ere were tpo-.'f^w., administrators wh 



who responded 



; ' * do vith who shQtj^ld be. i^^v^l^e^.as 'participants^^ in the various phases c^f evalj^ationj 

example, the fie *- ..jin^ — x.i — ^.i.^ <r.-i ^ — ^ a 

Xivolva; Board 

/ yete' less willing-' t:B>o';:tf^^^ edupa.tots to 'invoiv6 Oom&^iltyriifeti^^ 



/ , For eMtnple, the Secondary .educators were more willing than\the Elementary educators . 

Involve' Board /Metib,^'j:^.. or .Student in.the. variqufe pliSs^s od^ It he' process. But; theV 



rs and'^Educa'^ , 



'^"tlonal Exp^y^fr/ jt^^^hf^';pQi^^ 'jenuTexi atn^.l|j^^ tochers, t^ie ' • 

r / ^ 'Z ^l^*. •'^^F^'^v^ .^''^S^?.^^^^^^^ .Tenured 



some: Bethod6iogl'qal ';in'&i:ghts .a^>ell as the factual 
Flr^t, .a stoal^Bilot stu^y'^shoiild ,have beejn conducted* ^^^^^v/^ / ^ 
di'ffe^?eAces, ill iBip'ortance ratings ynd'e^r the two- aasuwpf'lorfaV. ' ' ' , Jf 




rating bpfh- * . , j 



.Impc^jtande an^' Effec^tvenes and" nj^i^lri'g ei^er an X'; or \a rating )^ to 5 

m \ *f% -««sii^s.. rThl^\Rrol?ably cqntM^olted* to mQasurerfent error because of some 



|j;j*and irr^itatlb|i*. ^It might b.e better. tct eliminate the "X? 





quite time- ^ e 
late April 
respondents to 



" 'v^t ' /'^V'f yii-Etih, j^faile-'thls-.'sujrvey '*does.help to identify areas ''of -discrepancy be- 

^'^"^.^tw^^ Effectiveness, it does not help the school people in deter- ^ • 

i ' nj^Tilng- exac;tiy what is wrong- with those discrepant policies or procedures. • We 

[ j^oul(i^^nooMioa§^''.^X:l^(& school administrations to dpvye some time in committee with ^ 

teachei^s:.or la-^to'e'-to-one interviews exploring thi^ specific parts of the system 
r^y ' ^^y '-/ /' '7' > " ' * . - 

. .. - - • - z^ '* ' ^/ .-^^ . 

y^'-^-'/feh^t are' notr .worfeing satisfactorily after they have been identified. 

^^A^^y A.. ; ^ ■ ^ ^ ; 

y' / ' ^-^ Finally, th4.s suir-vey. or its successor 'should be 'S'tudied and analysed with 

''^^ './y' ^reeard-to several obvious technical points. These would ihclude factorial validity, 

"^ut^test flWcgeneity, ^.tem-scale correlations , ' Subtest- orthogonality, test-retest 

■r "•.>■'..•.>, '■ '.-'. y . - . ' ■ - ' - • , ■• 

V .V ..'^r^iafi'iiity, and the like. ^ ' ■ ' ' ' • ■ 
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RESPONSE INSTRUCT! ONS 



ll^EM CONTENT ^ " ^ 

Pari; I consists of a number of items eacK of which describes 
either a ."procedure or a policy whi^h can be employed; In the^ 
development or operation of an evaluation system; 



Examples 



A Policy Emplo;^ed: in 'Development^^ 
..^(Item)j , Involve' teachers' 'in the selecting and 
defining- of the ediicatd'oTial objectives uporS 
which teachej^ evaluations are to be based. ' 
. I'' . . 

ir.e_Qjf O peration 



(Item) Have each observational visit long enough 
to allow the observer to get a' clear picture of what, 
the teacher is doing • ' * 

RATING TASK ..vr.- 
You will b& asked to rate each, item in- two different way^« .'pl'r 
you are. to rate, the item in tferms of how' important^^you . believe 
is to include the policy or procedure described in^the.item in 
any hew. evaluation system we might develop^ Secondly, you^are 
being asked to rate our pre'sent evaluation system in terms" of 
how effectively it handles, the policy or procedure .described 
i^nj^the item. - ^ y ^ , 

^ ' • * ' . . 

SPECIFIC RATING ^DIRECTIONS * - ' \ 

1» Read each item' carefully. - ' — 

^^2*. • Decider in your own mind how important the ^procedure or 

policy des.cribed in the item is. - "f J^^ V ^ y 

5. Look at the Importance , sea le^ on ^the ' top /of \the response 
sheet to find the 
. yqut feelings about 

4. ^ Place this number , in fhe squa're/P^' ^^i,cTi: appear^ to the 

^^ft of the item.' Use "x^ if item should not be cTonsidered 

5. / Now think about how ..effectively our^^resent system d'e^als 
, . withw the* policy or procedure;*' ^ \^ 

6. . Look at the Effectiveness sea'le on the top of theXresppnse 

sh'e^t. and^firld the rat ingt'-( number) .which .best rep^eserits'^^ 
your* feeli,rigs. ^, ?> ' ' * . ' 

7. ^ Place that number int the circle ^T^wJiicJi appears to t^^e ^ ; 

right of the item you ar^. rating. . Mark an "X** in the 
circle if the policy or ^p^ocedure "is"' not -practice<5T , ^ * - - 

8. ^ Go to. each successive' i;feem until'you have .completed the.-'/ 

'questionnaire. i. ' ' ' / .^ •^ * ' . / 

'9.' yhen yQu h^eV f iriish^<^V/chec]f your Response sheet to. ' . 

^Tiiake sure you have rat^d^ ^yeiry item twice'.: See. if you'- 
• " have pla^ced^ a number Vin both, the square and 1ihej.circle 
*for all the itemsr.- / . ' ^ 

' . 17 * 



rating (number) whidhj;i)est describes ' 
t the itemVs inrpor.tafnce. . " ■ 



" Sf'-. ' ^- CAUTION '\ 

Before you start '.responding we would like' to poi^nt out thr^e ^ 
rtXings to you. "First, your Impb^tance ratings ^^^ill be lis^d ^ 
to determine what policies and procedures ^re the most' crucial- 
to include in any n^w evaluation system.. Second", your teelifigs 
abbut Effectiveness will id^iritify those procedure^ artd 
policies .wltli 'which our present system has trouble.; .This 
will, tell »us .what types of changes are needed in^'any \ 
.modifications for improvement of • our presentT .system. 

. > J. ' ■ 

Third,^a technical' note : If you- place a lot of items in just 
^.arie or t^o categories (for ex^ample/ rate most items 4; or 5} 
yqu will, iiv/eff^ct, be diluting your own v^ote on makang 
these .djp'cisions. One way to avoid' this and -^t a make -sure 
your responses .will have ^ a maximum influence on the sp ^change 
'decisions is to try to place at. least four items from the , 
entire guestionnaiire in each of the five rating categories, ' 
both ^ for 'importa^nce and f or-'Ef fectiveness. 

When responding to all parts of the questionnaire as:sume that 
you will have/ approximately your .present 'amount of influence 
upon such i^'ctors in the claBsfobm as , class size, -student 
arrangements, materials, other resources, etc. that influence 
student .at nd teacher 'behav^Lors. /• • - 




. PART i 
Section 1 



To the Respondent : 

The. responses you make, iin this 8e(^ion cou^ be, used to •fcjrm • 
an evaluation systerh which would[)|jrovide infdmatibn solely for 
the use 'Of individual teachers ii\, improving instruction.^ . 
With .this' assxmptian in mind sta.:^t with item and give your 
•ratings 'of Importance and' Efffecti^ness . 



/ 



A, second section will ask you tq respond assuming that the 
evaluation system^ woulxi be.\^pd to" provide ihfomation to 



administrators. This informatiqh could 'be used by them for. 

ry^^ \/ - • ■ . . ' ~ ' 

niaking su9h iiecisions a.£f^ Promotion) " Retention-Disrpissa^l/ 
' ' ' ■ ^ " - ^ . ^ * ' ^ / 

Tenur^-Non Tenure, .Merit Considerations and, Job Assignments. 

Remember, when responding to this settion assume that you will \ 
have approximately your present amount of. influence upon Such 
fadtors im the classroom as clasfe siz6% student arrangements, 

matei>a;als, other resources, e.tc. that influence student and 

/ . . . " ^ ^ . ' 

.teacher behavifcrs." ' 
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A/ S'Slectinq and Definincr Eduoational Objectives 

.The term educational objectives inclaHes both instructional and 
professional objective^ Insbriictional objectives can be defined 
as desirable changed in students' knowledge and behaviqr. 
Professional objecti^ves include such things, as t^eachers' 
self 'improvement , ' assisting other teache?:s in increasing their 
effectiveness or carrying , ovit established administrative policies, 

Educational objectives could Ipe employed as the basis "for 
jleciding what will be observed 'or measui^ed iX order/to ^, 
evaluate tea,ching». With this in ^lirld, indicate on the response^^. 
sheet how important it is to and'how effectively has] our^' ^ 
present system been. .able to:" / - 

1* Develop a written list of 'pacifically defined . 
objectives tovyard "wljiich every t^'acher is expected 
■ ^ to work. ' ' , ^ . , 

2* Develop addirtinonal objectives which are more specif ic ^ 
* ' to grade ' level 'and^ subject or departmen^ial areas*- 

3-9. Include representatives pf^ th'e following groups in 
' I ' establishing objectives/ * « * • " * ' • ' 

3.. teadhers' j / ^ , I * . / ' / , ^ - 

' - ' 4. administrators y : \ ^ , 

• ,5. student3 ; / ' ; : • 

^ ' 6. parents ■ . >^ ' ' ^ ' 

7. community members , * ^. . - . - 

8. school board members ^^f^ , ^ • ^ 
. * 9. educational experts 

-^10^.' Provide the teacher witl> av written list of genef'^l • 
objectives for , all/ teachers -and specific /"objectives 
.tot teachers at a^given grade level or-in a "particular 
^ -subj'^ct area, y / ' ' " 

11. Give different w;eights to different objectives (r.e. 
'assign priority to them/ on the basis of a rating 

inventory (lik^ the one. ybii. are taking) given to 
all t e ache r s^. / ' . . ' - ' 

12. ^ Allow teacher^s to add to the /general list of ; ' 

objectives any .they p.ersona.lly • feeX 'are imp^6rtant .\ 

13. ^ Provide -an i^n-service session for clarifying' the ' 
' , general list of pbjedtiyes. . , *• ' ' '■ : , 

lAi Provide a 3:fegular- and .systematic way to i?eview * 
and revise the lis-^ .of object ives.^^ 
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De^tfeTtfainincj Xi^^riables 



)e EMplo^ed in Evaluation 



Tfi^re are basi\cally only twd deans of assessing a' teac]i^r*s 
ability tb achieve a given* educational objective* 'It can be 
determined eithe'r.by a proceks ' evaluation or ^a product / 
evaluation^. Process evaluation focuses on what a teacher 

instruction while t)roduct ^ 
t^jident behaviors which' 
ically should be affected \as a result of good instru'ction. 

It \s possible and perhaps desirabie to employ a corqibinatipn 
of ubt-h fotms of evaluation. 



^oe's ia order to provide goc 
alua'tion focuses on .tho3e 



witK this in mind, indicate 
fectivel^ has our" present 



how Ipaportamt dt is and how-^.e 
* system^ been able to? 

\\ 

1-7% InC|l.ude ^r,epresentatii/es from the ^ following" groups 
• ' 'Win selecting the variables to be used ii;i assessing 
- , ' ^\ teachers* 




10. 



11. 



12^ 



13, 



ability ^ 
teachers 
administrators 
stud'e nts 
parents . 
community !?iembeT:sj 
school board members 
educatioaaX experlts 
Pr'&vide an in-service 
"variables ^elected wi 
(Objective by object! 



meeting at (Which each of 'the 
11 be clarified and' justified* 



Alld^ -individual, teachers .to add to the variables 
s e le pt ed , a ny ; add i t i o 
also: be appropriate 



nkl ones which they feel would 
iin^assessing their abilitiy. 



ic'way of reviewing and-^revis-ia^ 

used- in evaluation, 
the use of p^rocess variables 



P r p vid e s bme sy s .t ^n^a t 
the. list of variables 
Evaluate teaching by 
, '(teacher actions) - 
Evaluate teaching by jthe use of prbduclPvariables 
(co^ise'guent student •abtions)* • ' , • ^ 
Let eat:h teaclier decide 'wljether-. his/her teaching. ^ 
is to be_ evaluated/by 1 Process or Product Variables, 
or both and try the choice for at least one year... 
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V 



^ ^ * 

Determ-ininq the rechniq u^ tQ- be E-gployed in Gathering 

Evalua^tive/lnfcmat ion ^ . * . ' ' . 



,3^ Basically there are^. only two means available for gathering 
data on the variables, ^vhi^ch are seL|cted. They are .the 
observational and irist't-im^ental techniques. 

pbserva;^i6nal techniques are measurements obtained from an 
. observer (Ratings.) who has watched, or listened to a teacher 
-at ^ork ' ("f or process variables)' or students' performance 
(for product v^$;'iable3) • ^ " ' • v - 

\ . , ' ' / ^ '/ ' ' ^ , 

The instrumental techni'cjue' involves measurements gbtained 
ftom sifudent tes'ts " (achie^vempht , attitude etc.)# surveys, 
questioiinai2ses, or some cth^ sort of instrument which 
sample3 directly a 'teacher ' s behaviors (process) or the . 
students ' beJiaViors (product) 



WitTi these definitions in mird indicate ho^ important it 
(is and how' effectively has ojr present system been able toj 



1-7: 



,8. 



10. 



11. 



12 i 



13-.' 



Include represej^tatives of tlie following groups 
in deciding' whether obs-ervations' and/or instrvunents 
'^houl,d' t)e used, in irjeasuring a given variablet ' 
, ll- ■tsea'chers ' • ' ' . ' ' . ■ 



'2. 

v. 

.4. 
.5. 

7. 



■t administrators. ' 

-> communi'ty members • 
scSt^ooI fe'Q^rd -members 
educational-^experts 



Provide ,.aii -in'-^service -session fo3j examining 'and 
,pnderstan5:ing\^ater i.als' 'and pr*6cedures to bemused 
r' ""in measiirLng. ^ *' • . ^ 

Allow teachers to go', beyond the limits, of the. 
particiilar tech n'iqqe - chosen and add measures they 



. believe t^o be. import;ant and wish ^o use» 4 
Pi?ovdd^Q*a tinte -and .a. system' ^f or reviewing and 
reATiaiug the decisions abput -m^thQds "for d^ta 
"fathering.''' ' * • 'm--'^ \ ' '"^ 

Evaluate teaching\^by means of obse^rvational ' -'- 

techn'iques» — — ~^ ^ .-^4t > , ' " / 

•Eva^iuat? , teactiinq by means lof instrumental 

Let each teacher decide whether his/he t teaching 
is-\cr be 'eval;:3afe^d by;, obseryatipnal. or instfumerital 
-techniques.'*cr botKand t^y the choice^ for at leasts 
ane'year.- ^ ' " - / - " ; ■ 



4 



/ V 

/ 



6r/ 



D > Guideline for Observational Assessment \ 

Assume the observatipnaJL technique ha^foeen .chosen to measure 
some variable, ebw important is it^ai^^how* effectively has 
^our* present system been .able to:' 

. 1-7* Include ■ re^resenta,tiv^s from the following groups in 
determining who^ will observe the teachers' or 
students' jDehavi'ors chosen- as variables, outline , 
the procedures to be followed by the observer, and - 
^ - ' develop or select the materials to be employed s 
1* teachers 

/ 2. administrators .... - . ► 

3* students \ • ' . ' ' 

4. parents / 
5« dommunity members i 
&• school board members 
^> ; 7« educational experts , 
8-1?/ 'Include members of the following groups as observers 
8o central office administrative staff 
9* principals' ^ \ * 

peers , . - 

f 11. self ^ ^ 

12. st"ydents\ ' 
>13.' parents ' • . ^ 

14*. community memfe^rs ^ . ; . ' ' * 

15. school board members 

16. . educataiona'l experts / " ■ 

17. others (please specify/ on respoi^se sheet) 
.,18. Provide fo£' sufficient number of fd'.rnial obse'rvations 

per term to obtain a precise measure on each variabl 

19. Provide, for a^sufficient amount^ of *time on each* 
formal observation to allow. the observer to get 

^ a clear^picture of what the teacher or students. is 
doing. . , 

20. -. Provide^ for sampling (formally observing)* the same 
behavior during di*fferent times of the day. " . ' 

2X» Allaw each teacher to schedule the day and time 
^ " . . .^of^tne majority of the'formal observations. ^ • 

Provide\a training p2?ogram for the observers. ^ 

23. Provide the Observer with enough background , 
^ . information. about th^- students (skill. level, etc.) 

or. teacher .^•tobjective-s'^ lesson plans,; etc.) to be 
* ^ ^ observed for him pr.her to underst4nd what is 

' going, on. • . . . »v / 

24. Provide', an in-service session' for trying out and. 
getting u'spd .to new modes of* observation (e.g. 
video tapes, ' etc.) . 

2 5... Provide a,- regular time- and proce.ss ^f^r reviewincr|^M 
i .and revising the observational programs (including, 
: who observes, materials us^d, procedures followed, 
and reports given.)!lf; : • - 



E, Guidelines for rnstraaiental Assessment 

Assuming* some instruments (achievement tests^ attitude ' 
tests, suryeys ^etc) are being employed to pleasure *one or 
mcyre variables, hew imporr^-nt is it and ^how effjectively has^ 
our present system been able to: " ' . • , 

1-1 • ' Include representatives from the following groups / 

in^ selecting and/or developing the instruments . 
to be used in measuring variables: ' . - 

1. teabhers 

■r 2. administrators ** ^ . - J 

3 . students ^ ■ , ' , ' 

4» pare^nts * 
^ . 5. . community members • • - . 1 

6. school board members 
7.. educational experts 
8* Make an effort to be sure that the tests to be used 
: ' are valid and reliable: 
9. Provide teachers with written examples of the items * 
contained in the instrumer^ts to be used* in assessments 
10. Allow individual teachers to add any items or subjects 
which they, think are important to olDtaini^g an, 
•-accurate measurement • . : 

J.1. Schedule an in-service session at which the tests, 

surveys, etc. would be explained, demonstrated, ' • 
' and defended. ' ^ ^ ^ 

1^. Provide a regular time and. system for reviewing '"^ " 
and/ revising the instruments -and theiir items. 



F. Deciding About Standar d s of Excellence in Teaching 

_ , ' • \ s ' ^ ' \ 

In any, evaluation, of teaching sysrem .it is^ possible to 
establish minimun*, standards wh'i.ch must be met and some 
graduations above the minimiim standard so that all jt^achers 
• can be adequately and fairly assessed. 

With this in mind indicate ho^,/important it is and how 
effectively our present system has been able tos * - 

1-7, Include representatives of the following groups in 
determining what are' minimum. standards and what 
"graduations should be determined above the mir^mum 
standards: • ' o • 

1. teachers • • * 

^2. administrators ^ \ ^ ' ^ 

3. ^ students ' \ ' ' 

4. parents ^ ' < ' " ' \ 

5. qommunity mem.bers ^ ^ \ . ' • . 

6. i^chool board members- . ■ \ . • 
7-. ' educational experts 

8. ' Provide a written statement about standards 'to teachers. 
' 9. Provide an in-service session for explaining and 

justi-fying all the standards o , ^ " 

10. Provide for ad -Lusting the^ standards for teachers with 
' • unusual class s^ize/ work loads, poorer or low 

achievers, gtc. ^ 

11. Provide f^r a regular sys^tem for evaluating and . • 

■ ' . revising ^standards. ^ - '/ 
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G* How ' Should Evaluartion lnformabion Be Shared • ✓ ^ 
■ ^ I ~ , " • ^ • ' 

This sectioYi re£ers to procedures wiiich could followed in *^ 
' order to provide iaformation to, the teachersS regarding the ^ , 
results of his/her evaluation. * * j — ^ , ^ . 

^ With this in mind tkll how^ important ±s it and how effectively 
-.has* the present system bee^ able to: ^ . " ' J ' ' ^ 



1. ' Provide for treque-nt' evaluation "sessions during the . 

•'year. ^ ' ■ ^\ . ' ^ ; 

2. Schedule feedback s,essions as soop as possible after,' 
each of fhese" evaluation sessions (whiTe things are 
still freshen people* s minds). ^ ' . . * 

. 3.' Have a written report to accompany the* feedback' ^ 
.•session (so. tha.t forgetting of ve rbir cemiftunidations 
is not a ^ problem). ' ' ^ " - "^'^ - 

4. Allow or even encouf^ge tlie indiyidiSal-tea'^her to"' . 
,prQvide additional evgiluative/ inf omation he or she 

h^s 'acc^Uired independently to corroboraA^g^ W, refute . 
th^ writter^\ report. , , * . 

5. Focus/part o\£ the * ses&ibn /upon possible changes' the 
teacher coUIol make ;(alterat<i:on of s'tyle. or a^prqach^ * 

" use oi differ^t materials, etc.) in' orcier to ' , > 
improve or vai^ rtistxuction. ^ ' V ^ . ' • 
6^. Provide th^ needed resourge materials and. i'^eas 

to help a teacher Who wishes to try- some innovative 
. * - approach to Improve :hts or her ability ^ \ ' 

7. . Pro>^ide a relaxed ati^hos^hexe^ that encourages • * • . 
" teachers to engage in self "^^^*ation and to take 
au .active part ih^dete mining th*e direction of^ the 
^ feedback 'session.'' • v: ' ^* - 

' * 8. Provide a system by .whi^ch the feejdb|ck- procedures**^ , 
. 0* can be changed when necessary.^ " • ' ' . • 



1 - " • ? 

. '"' \ PART I • ' • . 

• ' ^ * ' Section' 2 / ' 

T o the Respondent ;. ' , ^ . . ^ • 

The responses you iT\ake ^in this section could be used to form an 
evaluation system which will provide information. to administrators. 
This information .could i>e; used by them for making, such ■ * 
'decisions as Promdtioa, Retention-Dismissal, Tenure -Nontenure, 
M^rit considerations " and job Assignments. ' - / 

When responding tp^this ^section assume that you will have 
approximately :^Our present amount of -influence upon such ' ' . 
factors in the classroom as class size, student arjrangements, " 
mat^'eri^ls, otber resources, etc/ that influence student ^ 
and teacher behaviors. ' 4 • . . ' 

We are asking you to rate th^ 'same items -as before — only 
wi^th the ^difference -in how .your ^rating^s may.be- used, .s&n 
go back to item Ai. and answer each item with, these iiew uses > 
in mind.^. ' , • . ' . 

^ ■ ^- .V ■ . » ' V '■ "• ■ . ■ ■ . .- 

.After you- have finished...Part I,, -go on to Part JI on the 
following page. it. is short and very important. ' . , 



. •• V ■•• ■ . ■ PART 11 

ft 

To 'the Respondent ; •' ' 



aluation system* They are.> instj^adv;';;:if|:^/' ^. A' ^ J,V' 
ion in'format;Lon. Tell ho> . iinpprj|:^u^^,'\^^^ > '.r^ . 
w effectively our sp'i>e bent systenv'hAsj^r ^" ; j 

teaching information' \o: • v' 'J/*' .'"'"^ '/^'Jl'v'' 



Tlie items in this final part are not rea 
procedures '-of an evaluation system* They 
v/ays to us^ evaluation 
each item 'is and how 
used evaluation of 



1. Help teachers improve the:|r teaching ef fed^^iy^.^^^ 

2. Reward Superior pe^rformance f inane iallyj^^.'Jjr'' ' y " - . 
.3. Determine .tenure.. , * i/ -'i^ - s ..-^ 
-4. Make decisions regarding /sta^f^n/assigrmiertt %' ^' '/-''^ 
,5- Make decisions regarding retkKl^uiQni and dis^is^aLV-:;:^^^^^ 

6^. .P!isotect. the teacher in legal . ' H.-:'^ b^/^"^ • 

?• Protect the* school district/ i'Vi^^'tal 
''8. ' Validate tKe instruments -andi^ sy^^^'s -used* ,:^;r. ' , ^.'^ ^. 

personnel^selection and ass^igm^^i;it-/' ' ^ -/ r'^rY? ' ^^ Y^ 
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